Systems and Methods for Detecting Complex Networks in MRI Image Data

ABSTRACT

Systems and methods for detecting complex networks in MRI image data in accordance with embodiments of the invention are illustrated. One embodiment includes an image processing system, including a processor, a display device connected to the processor, an image capture device connected to the processor, and a memory connected to the processor, the memory containing an image processing application, wherein the image processing application directs the processor to obtain a time-series sequence of image data from the image capture device, identify complex networks within the time-series sequence of image data, and provide the identified complex networks using the display device.

CROSS REFERENCE TO RELATED APPLICATIONS

The instant application claims priority to U.S. Non-Provisional patentapplication Ser. No. 16/368,774, filed Mar. 28, 2019, which claimspriority to U.S. Non-Provisional patent application Ser. No. 15/997,631,filed Jun. 4, 2018 and issued as U.S. Pat. No. 10,285,658 on May 14,2019, which claims priority to U.S. Non-Provisional patent applicationSer. No. 15/820,338, filed Nov. 21, 2017 and issued as U.S. Pat. No.10,034,645 on Jul. 31, 2018, which claims priority to U.S. ProvisionalPatent Application No. 62/485,196, filed Apr. 13, 2017, U.S. ProvisionalPatent Application No. 62/563,611, filed Sep. 26, 2017, U.S. ProvisionalPatent Application No. 62/568,676, filed Oct. 5, 2017, and U.S.Provisional Patent Application No. 62/589,452 filed Nov. 21, 2017, thedisclosures of which are hereby incorporated by reference in theirentirety.

FIELD OF THE INVENTION

The present invention relates generally to image processing, includingautomated image processing, and more specifically to the detection ofobjects and/or signals within images.

BACKGROUND

Object detection is a key component of many image processing andcomputer vision systems that enable a computer to locate and classifyobjects within an image or sequence of images. Classic examples ofobject detection include facial recognition, and object tracking.Processes for automated image tracking can also be useful for detectingobjects or patterns that might not be immediately apparent to humanviewers of the image. The ability for computers to perceive andcomprehend visual data is key for enhancing the capabilities of computersystems to interact with, and provide feedback on, their environment.

SUMMARY OF THE INVENTION

Systems and methods for detecting complex networks in magnetic resonanceimaging (MRI) image data in accordance with embodiments of the inventionare illustrated. One embodiment includes an image processing system,including a processor, a display device connected to the processor, animage capture device connected to the processor, and a memory connectedto the processor, the memory containing an image processing application,wherein the image processing application directs the processor to obtaina time-series sequence of image data from the image capture device,identify complex networks within the time-series sequence of image data,and provide the identified complex networks using the display device.

In another embodiment, the time-series sequence of image data includesdata describing a set of images taken from a specific viewpoint overtime.

In a further embodiment, images in the set of images are threedimensional images.

In still another embodiment, identifying complex networks within thetime-series sequence of image data includes preprocessing thetime-series sequence of image data, detecting structures within thetime-series sequence of image data, and measuring connectivity betweenstructures within the time-series sequence of image data.

In a still further embodiment, preprocessing the time-series sequence ofimage data includes realigning the time-series sequence of image data toa fixed orientation, unwarping the time-series sequence of image data,and despiking the time-series sequence of image data.

In yet another embodiment, a magnetic resonance imaging image processingsystem, including at least one processor, a memory connected to the atleast one processor and containing an image processing application, adisplay in communication with the at least one processor,synchronization circuitry, a magnetic resonance imaging machine incommunication with the synchronization circuitry and the at least oneprocessor, wherein the magnetic resonance imaging machine is directed toobtain image data describing at least a patient's brain, and a stimulusdevice connected to the synchronization circuitry, wherein the stimulusdevice is configured to provide a stimulus to a patient, wherein theimage processing application directs the processor to obtain the imagedata from the magnetic resonance imaging machine via a network, generatea time-series sequence of image data of the patient's brain, wherein thetime-series sequence of image data is time-stamped with the times ofstimuli provided by the stimulus device using the synchronizationcircuitry, and pre-process the time-series sequence of image data toidentify brain regions, generate at least one neurological model, the atleast one neurological model including a data structure describing atleast one network within the time-series sequence of image data, andmetrics specifying neurological activity observable within thetime-series sequence of image data, assign a biotype to the patientbased on the neurological model, and provide a graphical user interfacecontaining the assigned biotype using the display.

In a yet further embodiment, preprocessing the time-series sequence ofimage data further includes realigning the time-series sequence of imagedata, unwarping the time-series sequence of image data, despiking thetime-series sequence of image data; and, registering brain structuresobservable within the time-series sequence of image data and a brainatlas, wherein the brain atlas comprises a data structure stored in thememory and describes physical regions within a reference brain.

In another additional embodiment, despiking the time-series sequence ofimage data comprises measuring displacement between images in thetime-series sequence of image data, and identifying time periods in thetime-series sequence of image data where the displacement between imagesis greater than a frame displacement threshold.

In a further additional embodiment, the atlas is the AutomatedAnatomical Labeling atlas.

In another embodiment again, at least one of the at least oneneurological model is a first level reactivity model, and the metricsspecifying the neurological activity specify strength of neuronalactivation across brain regions.

In a further embodiment again, the image processing application furtherdirects the processor to generate a first level reactivity model,wherein generating the first level reactivity model includes extractingimage data describing brain structures from image data describingnon-brain structures in the time-series sequence of image data, removingglobal signals from the image data using a white noise mask, spatiallysmoothing the image data using a blurring kernel, and determining adegree of neuronal activation in response to stimuli deliveredcorresponding to the time-stamps in the time-series sequence of imagedata.

In still yet another embodiment, the blurring kernel is an 8 millimeterblurring kernel.

In a still yet further embodiment, the at least one of the at least oneneurological model is a psychophysiological interaction model, and themetrics specifying the neurological activity specify degree ofconnectivity between brain regions.

In still another additional embodiment, the image processing applicationfurther directs the processor to generate a psychophysiologicalinteraction model from the pre-processed time-series sequence of imagedata, wherein generating the psychophysiological interaction modelincludes performing slice time correction on the pre-processedtime-series sequence of image data using a descending interleavedacquisition sequence, normalizing the pre-processed time-series sequenceof image data to a coordinate system, spatially smoothing thepre-processed time-series sequence of image data using a blurringkernel, defining at least one volume of interest in the pre-processedtime-series sequence of image data using a mask, extracting aneigenvariate from the at least one volume of interest, generating avolume of interest data structure including a deconvolved time courseincluding the physiological component of the psychophysiologicalinteraction model, a psychological variable describing a parametriccontrast of task onsets from a contrast of interest, and data describingthe interaction between the physiological and psychological variables.

In a still further additional embodiment, the at least one neurologicalmodel comprises a resting state model.

In still another embodiment again, generating the resting state modelincluding concatenating preprocessed image data across stimuli,segmenting the concatenated image data by tissue types, generating atleast one regressor matrix, generating residual images from thesegmented preprocessed image data, and bandpass filtering the residualimages.

In a still further embodiment again, the tissue types are at least whitematter, grey matter, and cerebrospinal fluid.

In yet another additional embodiment, the at least one resting statemodel is a region of interest resting state model.

In a yet further additional embodiment, the at least one resting statemodel is a voxel wise resting state model.

In yet another embodiment again, generating the voxel wise resting statemodel further includes extracting a time-series sequence of image datacorresponding to a particular set of stimuli from the segmentedtime-series sequence of image data, and applying regression against theextracted time-series sequence of image data for a particular set ofstimuli against the segmented time-series sequence of image data for allvoxels, indicating the overall resting state for each voxel.

In a yet further embodiment again, assigning a biotype includesobtaining a database of biotype classifications including image dataannotated with reactivity and connectivity metrics associated withspecific biotypes, matching metrics from the resting state model to thebest fitting biotype classification.

In another additional embodiment again, the database of biotypeclassifications is generated using machine learning on a training dataset.

In a further additional embodiment again, assigning a biotype includesgenerating an indicator of fit describing how closely the biotypematches the patient.

In still yet another additional embodiment, the assigned biotype isassociated with at least one treatment, and the image processingapplication further directs the processor to recommend the at least onetreatment to the user.

In another embodiment, the at least one treatment is an at least onedrug, and a medical professional treats the patient by prescribing theat least one drug to the patient.

In a further embodiment, the image processing application furtherdirects the processor to generate at least one efficacy metric based onthe assigned biotype, wherein the at least one efficacy metric indicatesthe likelihood of success of treating the patient using the at least onedrug.

In still another embodiment, the magnetic resonance imaging machine isconnected to a first communications port capable of transmitting dataover a network, the processor is connected to a second communicationsport capable of receiving data over the network; and the magneticresonance imaging machine and the processor are connected via thenetwork using the first communications port and the secondcommunications port.

In a still further embodiment, the stimulus device provides a Go No-Gotest battery and an emotional regulation test battery to the patient.

In yet another embodiment, the stimulus device is a visual displaydevice.

In a yet further embodiment, a method for identifying complex networksin time-series sequence of image data, includes obtaining time-seriessequence of image data from an image capture device, realigning thetime-series sequence of image data to a fixed orientation using an imageprocessing server system, unwarping the time-series sequence of imagedata using the image processing server system, despiking the time-seriessequence of image data using the image processing server system,detecting objects within the time-series sequence of image data usingthe image processing server system, measuring connectivities betweendetected objects within the time-series sequence of image data using theimage processing server system, matching the measured connectivitiesbetween detected objects to at least one complex network stored in adatabase of complex networks using the processor, and displaying the atleast one matched complex network using a display device.

Additional embodiments and features are set forth in part in thedescription that follows, and in part will become apparent to thoseskilled in the art upon examination of the specification or may belearned by the practice of the invention. A further understanding of thenature and advantages of the present invention may be realized byreference to the remaining portions of the specification and thedrawings, which forms a part of this disclosure.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a system diagram illustrating connected components of an imageprocessing system in accordance with an embodiment of the invention.

FIG. 2 is a flow chart illustrating a method for detecting andclassifying objects in images in accordance with an embodiment of theinvention.

FIG. 3 is a flow chart illustrating a method for performing a testbattery in accordance with an embodiment of the invention.

FIG. 4 is a flow chart illustrating a method for providing stimuli inaccordance with an embodiment of the invention.

FIG. 5 is a flow chart illustrating a method for preprocessing imagedata in accordance with an embodiment of the invention.

FIG. 6 is a flow chart illustrating a high level overview of a methodfor generating reactivity models in accordance with an embodiment of theinvention.

FIG. 7 is a flow chart illustrating a high level overview of a methodfor generating connectivity models in accordance with an embodiment ofthe invention.

FIG. 8 is a flow chart illustrating a method for preprocessing data forconnectivity analysis in accordance with an embodiment of the invention.

FIG. 9 is a flow chart illustrating a method for generating PPI modelsin accordance with an embodiment of the invention.

FIG. 10 is a flow chart illustrating a method for modeling resting statein accordance with an embodiment of the invention.

FIG. 11 is a graphic illustrating several neural circuit pathways inaccordance with an embodiment of the invention.

FIG. 12 conceptually illustrates several biotype neural circuit pathwaysin accordance with an embodiment of the invention.

FIG. 13 is an illustration of various biotypes in accordance with anembodiment of the invention.

FIG. 14 is an example of a survey rankings in accordance with anembodiment of the invention.

FIGS. 15A, 15B, and 15C are example assessment score reports inaccordance with an embodiment on the invention.

FIG. 16 is an illustration of various neural circuit pathways inaccordance with an embodiment of the invention.

DETAILED DESCRIPTION

Turning now to the drawings, systems and methods for detecting complexnetworks such as (but not limited to) neural circuit pathways in imagedata are discussed. Further, methods for complex network detectionwithin a three-dimensional (3D) matrix of pixel data are disclosed.Object detection within images is a key aspect of machine vision, andallows computers to parse and understand image data. In a time orderedsequence of images (video), object detection over the video can be usedto track activity, and enable detection of complex networks. Further, ina larger system where other components can trigger changes in theobserved image, time synchronization circuitry can be used to matchsystem activity to observed responses in a series of images.

Enhanced object detection capabilities have numerous applications,including, but not limited to, automation of tagging and classificationof objects within images, as well as images themselves. The ability toclassify objects allows a computer to perform additional processing,extract additional data, and/or utilize image information in otherprocessing operations. In some cases, object detection can trigger imageprocessing systems to produce warnings, reports, and/or otherhuman-interpretable signals to automatically notify a user.

Network detection methods described below can be used withunconventional image capture systems, and can further be used to allowcomputer systems to process data in new and unconventional ways. Innumerous embodiments, object detection and classification are used togenerate a taxonomy of objects. Taxonomies of objects can then be usedfor rapid analysis of future images. In some embodiments, machinelearning techniques can be applied to generate taxonomies.

One of ordinary skill in the art will appreciate that while specificimplementations of methods and systems for performing complex networkdetection are described below, there are numerous embodiments inaccordance with the spirit of the invention. While specific discussionwill be directed to performing complex network detection within MRIimage data, the systems and methods described below can be used todetect complex networks in a variety of types of image data and/orapplications.

Image Processing Systems

Image processing systems can be used to acquire image data, processimage data, and display processed data. In numerous embodiments, imageprocessing systems are constructed of multiple computing systems. In avariety of embodiments, image processing systems are implemented on asingle computing system. Image processing systems can process a widevariety of image data, however certain specific embodiments can beutilized for processing MRI image data.

Turning now to FIG. 1, a system diagram of an image processing system inaccordance with an embodiment of the invention is illustrated. Imageprocessing system 100 has at least one image capture device 110. Imagecapture device 110 is connected to image processing server system 120and image processing interface device 130 via network 140. In manyembodiments, the image capture device is an MRI imaging device. However,the image capture device can be any device capable of capturing an imageas appropriate to the requirements of a given application.

The image capture device can include various peripheral devices,including terminals, display devices, and other interface devices. Theimage processing server system can be implemented on a personalcomputer, a server computer system, or any other computing device asappropriate to the requirements of a given application. The imageprocessing interface device can be a personal computer, a tabletcomputer, a smartphone, a monitor, and/or any other device asappropriate to the requirements of a given application.

Image processing server systems can include a processor, memory, and/orat least one storage system containing an image processing applicationthat includes machine readable instructions that configures the computerto process image data in accordance with methods described below. Insome embodiments, the image processing interface device and the imageprocessing server system are on the same platform. The network can be,but is not limited to, the Internet, a local area network, a wirelesslocal area network, wide area network, a software defined network,and/or any other type or combination of types of network as appropriateto the requirements of a given application.

Devices described above can communicate via the network viacommunications ports. In many embodiments, data is transferred betweenone or more devices via the network. In a variety of embodiments, datatransfer between one or more devices is achieved using physical mediatransfer, such as a flash drive, compact discs, or any other physicalstorage media as appropriate to the requirements of a given application.

Once images are obtained by the image capture device, image datadescribing the captured image can be sent via the network to the imageprocessing server system for analysis. In some embodiments, image datais also sent to the image processing interface device. In numerousembodiments, the image processing server system processes received imagedata and outputs results to the image processing interface device. In avariety of embodiments, some processing is done by the image processinginterface device.

Processed data and/or any other output of the system can be provided tothe user by a user interface device. In many embodiments, user interfacedevices provide graphical user interfaces which enable a user to accessdata. In many embodiments, the user interface device is connected to thenetwork.

In many embodiments, one or more of the devices on the networkincorporate, or have access to, synchronization circuitry.Synchronization circuitry can be used to synchronize image capture withprovided stimuli. In numerous embodiments, the image processinginterface device is configured to provide stimulus using a stimulusdevice. In a variety of embodiments, stimulus devices are controlled bythe image processing server system, and/or the image capture device. Astimulus device can be a visual display device, such as, but not limitedto, a monitor, a virtual reality headset, a projector, and/or any othervisual display device as appropriate to the requirements of a givenapplication. A stimulus device can be an audio display device, such as aspeaker. Stimulus devices can provide visual stimuli, audio stimuli,tactile stimuli, and/or any other stimuli as appropriate to therequirements of a given application.

While specific network configurations have been described above, one ofordinary skill in the art can appreciate that any configuration ordevices could be used as appropriate to the requirements of specificapplications. Methods for image processing using image processingsystems are described below.

Performing Complex Network Detection in MRI Image Data Using ImageProcessing Systems

MRI scans of a patient's head yield high quality image data of thepatient's brain in a non-invasive manner. While there are many differenttypes of MRI scanning techniques, two categories of MRI scans are thefunctional MRI (fMRI) and the structural MRI (sMRI). sMRI scansgenerally acquire 3D image data of a patient's brain, defining thestructural anatomy. In many embodiments, sMRI scanning can includedifferent imaging techniques such as, but not limited to, diffusiontensor imaging, or any other specialized scanning technique. fMRI scansare a type of MRI scan that yields time-series sequence of image datadescribing neuronal activation within the patient's brain. In manyembodiments neuronal activation patterns can indicate the degree ofconnectivity between different regions of the patient's brain. In manyembodiments, MRI data is preprocessed to remove artifacts. In a varietyof embodiments, neuronal activation patterns can indicate the degree ofreactivity to particular stimuli in the patient's brain. Imageprocessing systems can be used to process fMRI data to extract dataregarding connectivity and reactivity. Based on connectivity and/orreactivity data, one or more biotypes can be assigned to the patient.Biotypes can be associated with one or more drugs and/or therapies thatcan be more effective in treating at least one negative symptom that thepatient suffers from. In many embodiments, machine learning can beutilized to associate biotypes with specific connectivities and/orreactivities. In a variety of embodiments, support vector machines areused to associate biotypes with specific connectivities and/orreactivities. Systems and methods for associating specificconnectivities and/or reactivities with biotypes are described below.

Turning now to FIG. 2, a process for performing complex networkdetection in accordance with an embodiment of the invention is shown.Process 200 includes (210) obtaining fMRI scan data, and preprocessing(220) image data. Reactivity metrics are generated (230), connectivitymaps are generated (240), and the observed connectivity forms the basisof a complex network classification enabling assignment of a biotype(250) to the observed neural circuit pathways. In some embodiments, atherapy can be recommended (260) based on the assigned biotype. In manyembodiments, process 200 is performed by image processing serversystems. However, in some embodiments, part or all of process 200 can beperformed by image capture devices and/or image processing interfacedevices. Methods for obtaining image data are described below.

While various processes for identifying and/or classifying complexnetworks, such as neural circuit pathways, are described above withreference to FIG. 2, any of a variety of processes can be utilized toidentify and classify complex networks using image data as appropriateto the requirements of specific applications in accordance with numerousembodiments of the invention. A number of processes that can be utilizedto identify and classify complex networks in accordance with certainembodiments of the invention are discussed below.

Obtaining Image Data

Image data can be obtained using specialized image capture devices, suchas MRI machines, that output MRI scan data. In many embodiments,patients are placed inside of MRI machines. Stimuli can be provided viaa stimulus device to the patient and their psychophysical responses canbe recorded using at least the MRI machine. These responses can berecorded over time using fMRI techniques to generate time-series data.In many embodiments, a resting state fMRI is generated. In a number ofembodiments MRI data taken during performance of tasks can be utilizedto filter MRI data obtained during periods in which tasks are not beingperformed to provide a more accurate model of resting state neuralcircuit activation.

Turning now to FIG. 3, a method for obtaining image data in accordancewith an embodiment of the invention is illustrated. Process 300 includesobtaining (310) a sMRI scan of a patient's brain. Process 300 alsoincludes synchronizing (320) at least one stimulus device with the fMRIdevice, performing (330) and fMRI scan of the patient's brain whileproviding (340) stimuli to the patient. In numerous embodiments, thesMRI scan is image data captured by an MRI machine. In a variety ofembodiments, the same MRI machine is used to perform the sMRI scan andthe fMRI scan. MRI machines can have ancillary components, includingconnected computing devices that collect and store data from thescanning components of the MRI. In many embodiments, performing an fMRIscan generates a time-series of image data depicting neural activitywithin the brain of the patient. The time-series sequence of image datacan be sets of sets of images, which together show the 3-D structure ofthe brain. In some embodiments, each image is a 3-D image representing aportion of the brain. A sub-portion of an image is called a voxel. Inmany embodiments, a single time-series sequence of image data set isgenerated during the course of an fMRI session. However, in numerousembodiments, several time-series of image data are acquired. In avariety of embodiments, at least one set of time-series of image data isacquired per stimulus battery (task).

Stimuli can be applied to the patient during scanning to measure theeffect of the stimuli on neuron activation within the brain. In manyembodiments, stimuli are images displayed to the patient. Synchronizingstimuli with the images in the time-series associates the response withthe stimulus that caused it. Synchronization of stimulus devices withMRI scanners can be achieved using synchronization circuitry. Innumerous embodiments, synchronization is mediated by a softwareapplication designed to associate continuous time series image data witha continuous time series stimulus regime. In this way, the image dataoutput by the MRI machine can be processed relative to specific stimuli.In a variety of embodiments, time-series data is generated that is timestamped, where the time stamps are synchronized with time stamps presentwithin image date captured by the MRI machine. In numerous embodiments,the delay between stimulus and the resultant image data time stamp is onthe order of 100 milliseconds. In order to correct for delay, stimulusimages can be jittered by +/−200 milliseconds. However, any jitteramount can be applied as appropriate to the requirements of givenapplications. That is, effects of stimuli detected within the image canbe correlated to the specific stimulus that caused the response.

While various stimuli and measurement techniques that can be utilized inthe detection of complex networks such as, but not limited to, neuralcircuit pathways are described above with reference to FIG. 3, any of avariety of stimuli and measurement techniques can be utilized asappropriate to the requirements of a given application in accordancewith many embodiments of the invention. Additionally, while systems andmethods utilizing MRI machines in accordance with various embodiments ofthe invention are described above, similar systems and methods can beapplied to image data obtained from other image capture devices, suchas, but not limited to, computed tomography scanners, positron emissiontomography scanners, or any other brain imaging device as appropriate tothe requirements of a given application. Methods and types of stimuliare described below.

fMRI Stimuli

Psychophysical responses are dependent upon stimuli provided. Particularstimuli, and the patient's responses can be used to generate diagnosticand treatment data. In many embodiments, diagnostic and treatment datadescribes a patient's biotype, and/or appropriate treatments tailored tothe patient's specific biotype. In many embodiments, correlatingparticular responses to specific stimuli can be used to generate data.In some embodiments, the particular sequence of stimuli is keptconsistent across tests for the same patient. In a variety ofembodiments, the particular sequence of stimuli is kept consistentacross many patients. The sequence of stimuli can be chosen in such away as to reduce the likelihood of artifacts

Turning now to FIG. 4, a method for providing stimulus to a patient inaccordance with an embodiment of the invention is illustrated. Themethod 400 includes performing (410) a Go No-Go test battery, andperforming (420) an emotional regulation test battery. In manyembodiments, motional test batteries include at least one face testbattery. Face test batteries can include, but are not limited to,conscious faces test batteries, nonconscious faces test batteries, orany other emotional regulation task as appropriate to the requirementsof a given application. Method 400 further includes performing (430) aresting state analysis.

In many embodiments, Go No-Go test batteries involve providing a seriesof representations to a patient as stimuli, wherein some representationsindicate a “Go” state, and some representations indicate a “No-Go”state. For example, a Go state can be a word, such as, but not limitedto, “Press,” and/or a color indicator such as, but not limited to,green. A No-Go state can be a word, and/or a color indicator such as,but not limited to, red. In many embodiments, representations aredisplayed at pre-defined intervals. In some embodiments, indicators aredisplayed for 500 milliseconds, with a 750 millisecond stimulusinterval. However, any display and/or stimulus interval duration can beused in accordance with the requirements of a given application.

Faces test batteries can include providing representations of facesrepresenting emotional states. Representations can be placed in aspecific, pseudorandom, or random ordering. Conscious face testbatteries can display the representations at intervals allowing forconscious recognition by the patient, such as, but not limited to, 500millisecond display times with 750 millisecond interstimulus delay.Nonconscious face test batteries can be designed in a similar way toconscious face test batteries with backwards masking. An emotional staterepresentation can be displayed for 10 milliseconds, followed by aneural (non-emotion state) representation for 490 milliseconds. However,any time intervals can be used as appropriate to the requirements ofgiven applications. For example, any set of intervals that allows fornonconcious recognition of emotional state representations can be used.

Resting state scans can be acquired by presenting the patient with ablack screen, and/or any other neutral, non-task stimuli, and performingan fMRI scan while the patient remains awake. While several tests havebeen described above with respect to FIG. 4, one of ordinary skill inthe art would recognize that any number of tasks and tests can be usedin accordance with the requirements of various applications.

One of ordinary skill in the art will recognize that the tests could beperformed in any order, and that no specific ordering is required.Further, not all tests need to be performed. In some embodiments, only aresting state analysis is utilized. Methods for preprocessing acquiredimage data are described below.

Preprocessing Image Data

Preprocessing image data can be performed by image processing systems toremove artifacts, and prepare the image data for subsequent analysis andprocessing steps. In numerous embodiments, preprocessing can be used tostandardize image data to known reference images. For example, inseveral embodiments, MRI image data can be preprocessed to yieldtime-series sequence of image data in which brain structures and/orregions within the preprocessed time-series can be identified by theimage processing system. In a variety of embodiments, a machine learningsystem is utilized to identify brain structures and/or regions using atraining dataset of brain images. In a variety of embodiments, the imageprocessing system uses a reference atlas to identify structures.

Turning now to FIG. 5, a method for preprocessing image data isillustrated. Process 500 includes realigning and unwarping (510) imagedata. Image data can be despiked (520) with respect to movement anddespiked (530) with respect to variance and/or any source of noiseartifacts as appropriate to the requirements of a given application. Inmany embodiments, spikes with respect to variance are the result ofextreme responses, such as, but not limited to, periods of very highactivity followed by periods of very low activity. Further, process 500includes applying (540) linear co-registration to identified structuresin the image data. Realigning and unwarping image data can be used toremove movement artifacts from a fMRI scan time-series of images.Methods for realigning and unwarping image data that can be used inaccordance with several embodiments of the invention can be found in theStatistical Parametric Mapping (SPM) library by the Wellcome TrustCentre for Neuroimaging of University College London, London, England.The manual of SPM version 8 (SPM8) can be found athttp://www.fil.ion.ucl.ac.uk/spm/doc/spm8_manual.pdf. However, otherprocesses, including other versions of SPM, such as SPM12 can be used asappropriate to the requirements of a given application.

fMRI image data can be despiked with respect to movement and variance.In many embodiments, the repetition time (TR) can be used to map whenspikes occurred. In a variety of embodiments, a frame displacement (FD)threshold is established and used to detect spikes. In some embodiments,an FD threshold of 0.3 mm is used. However, any of a variety of FDthresholds can be used in accordance with various applications of givenembodiments of the invention. In numerous embodiments, default variancecut-offs, and spike regressors are generated. Variance cut-offs and/orspike regressors can be utilized when performing later regression steps.In many embodiments, the cut-offs are used to ensure quality data forregression models.

Further, quality control (QC) images and metrics can be generated. Forexample, scaled and unscaled mean variance images across the time seriescan be generated. In some embodiments, max variance images across thetime series can be generated. In a variety of embodiments, temporalsignal to noise ratios can be measured across the time-series. Whilespecific QC images and metrics are described above, any number ofdifferent QC metrics could be used in accordance with givenapplications.

Linear co-registration can be performed using the FSL software libraryof by the Oxford Centre for Functional MRI of the Brain of theUniversity of Oxford (FMRIB), Oxford, England, including the FMRIB'sLinear Image Registration Tool (FLIRT). An online manual for the FSLsoftware package, including FLIRT, and a variety of other programsdiscussed herein, can be found at(https://fsl.fmrib.ox.ac.uk/fsl/fslwiki/). While a specific method forpreprocessing image data is described above, any number of steps couldbe incorporated for preprocessing image data in accordance with therequirements of a given application. Preprocessed data can be used in avariety of ways, including, but not limited to, generating reactivitymodels and connectivity models. Methods of generating reactivity modelsare discussed below.

Generating Reactivity Models

Reactivity models can be generated from preprocessed image data.Reactivity models indicate which regions and/or structures of the brainbecome active and/or reactive in response to a stimulus. In numerousembodiments, reactivity processing is performed region by region in thebrain. Reactivity processing can involve generating first levelreactivity models. First level reactivity models can include datadescribing the reactivity of particular regions of interest (ROI) in thebrain. ROIs can be individual structures, and/or groups of structures.In many embodiments, reactivity models describe the degree of activitywithin various regions of the brain. In several embodiments, the degreeof activity in a region can be classified as hypoactive, typical, orhyperactive. The activity of particular regions can be utilized todetermine effective treatments. Treatments can affect the activity ofparticular brain regions. In a variety of embodiments, additionalprocessing steps are utilized to prepare the preprocessed image data foranalysis.

Turning now to FIG. 6, a method for generating reactivity models inaccordance with an embodiment of the invention is illustrated. Process600 includes obtaining (610) preprocessed image data. Brain data isextracted (620) and global signals can be regressed (620) out using awhite matter mask. The image data can be spatially smoothed (630) andfirst level reactivity models can be generated (640).

In many embodiments, preprocessed image data is obtained from the memoryof an image processing server system. However, preprocessed image datacan be obtained from a variety of storage systems, such as, but notlimited to, flash drives, random access memory, hard disk drives, solidstate drives, SD cards, or any other storage medium as appropriate tothe requirements of a given application. Brain data can be extractedfrom preprocessed image data in such a way that non-brain tissue can beremoved from the image. In a variety of embodiments, the BrainExtraction Tool (BET) of the FSL software package can be used to performbrain extraction. However, any of a variety of structure detectionmethods can be used to extract non-brain material from images,including, but not limited to, edge detection, greyscale matching,gradient matching, corner detection, ridge detection, Hough transforms,structure tensors, feature description algorithms, and/or any otherfeature detection algorithm as appropriate to the requirements of givenapplications.

Global signals can be removed via regression using a white noise mask.In a variety of embodiments, the white noise mask is warped into thenative space of the image data being processed. The image data can besmoothed by applying a blurring kernel across each image in thetime-series. In a variety of embodiments, a blurring kernel of 8millimeters is utilized. However, any number of different kernel sizescan be used as appropriate to the requirements of a given application.In a variety of embodiments, nuisance regressors are applied to removeadditional spikes. First level reactivity models can be generated fromthe image data. In many embodiments, first level reactivity modelsdescribe neuron activation, including varying degrees of activation suchas hypo, typical, and hyper activation in response to specific stimuli.

While a specific method of generating reactivity models is illustratedabove, one of ordinary skill in the art would appreciate that therecited processes could be performed in different orders, or someprocesses may be added or omitted as appropriate to the requirements ofa given application. In addition to generating reactivity models,preprocessed data can be used to generate connectivity models. Methodsfor generating connectivity models are described below.

Generating Connectivity Models

Connectivity models can describe the connections between various regionsof the brain. In some cases, neuronal connections between regions areunderdeveloped, typically developed, or overdeveloped. In a variety ofembodiments, connectivity models and reactivity models describe complexnetworks. However, complex networks can contain information from justreactivity models, or just connectivity models in accordance with therequirements of a given application. In several embodiments,connectivity models are generated from preprocessed image data. In manyembodiments, preprocessed image data can be passed through additionalconnectivity preprocessing steps to enable creation of accurateconnectivity models from captured MRI data. Various connectivity modelsand metrics can describe how signals from different regions and/orstructures of the brain of the patient are transmitted.

In many embodiments, the organization of the connections between variousbrain structures and/or regions can be used to indicate responsivenessto various psychoactive chemicals. Connectivity models and metrics caninclude psychophysical interaction (PPI) models, ROI resting statemodels, and voxel wise resting state models. PPI models can describeconnectivity between a ROI and other brain regions, thereby indicatingthe brain regions where the activity depends on the psychologicalcontext and the physiological state of the ROI. Resting state models areused to estimate the resting state of a particular ROI, or a regiondefined by a voxel or set of voxels. Methods for performing connectivityanalysis are described below.

Turning now to FIG. 7, a method for performing connectivity analysis inaccordance with an embodiment of the invention is illustrated. Process700 includes obtaining preprocessed image data. Preprocessed image datacan be obtained in a similar manner as describe above with respect toreactivity models. Process 700 further includes generating (720) PPImodels, and performing (730) secondary processing for resting statemodeling. ROI resting state models can be generated (740) and voxel wiseresting state models can be generated (750). Methods for performingadditional connectivity preprocessing steps are described below.

Connectivity Preprocessing

Preprocessed image data can be passed through additional connectivitypreprocessing steps to enable accurate measurements of time-series data.Connectivity preprocessing can include mapping the image data to acoordinate system in order to define regions of the brain. Turning nowto FIG. 8, a process for performing connectivity preprocessing inaccordance with an embodiment of the invention is described below.

Process 800 includes obtaining (810) preprocessed image data. Slice timecorrection can be performed (820) and image data can be normalized (830)to a coordinate system. The image data can further be smoothed (830).

In many embodiments, slice time correction can be achieved using a givennumber of slices with a repeat time (TR) of 2.5 seconds and a descendinginterleaved acquisition sequence. In some embodiments, the given numberof slices is 40, however any number of slices could be used inaccordance with the requirements of a given application. Slice timecorrection can help correct for error introduced by non-simultaneousslice recording during a scanning iteration. Further, when the imagedata is normalized to a coordinate system, a variety of coordinatesystems can be used. In many embodiments, the Montreal NeurologicalInstitute (MNI) coordinate system is used. However, numerous othercoordinate systems could be used, including, but not limited to, theTalairach coordinate system. Additionally, spatial smoothing can beperformed similarly to the methods described above with respect torelativity analysis. In a variety of embodiments, once connectivitypreprocessing has occurred, connectivity models can be generated.Furthermore, first level general linear models (GLM) can be initialized.Methods for generating PPI models are described below.

Generating PPI Models

PPI models can be used to determine which neuronal activations in abrain at a given time are dependent on a given stimulus. PPI analysisgives insight into the functionality of an individual's brain. In manyembodiments, the FEAT program of the FSL software package can be used togenerate PPI models. However, any number of programs and/or packages canbe used to generate PPI models as appropriate to the requirements of agiven application.

Turning now to FIG. 9, a process for generating PPI models in accordancewith an embodiment of the invention is illustrated. Process 9 includesobtaining (910) preprocessed connectivity image data. The eigenvariateof the time series is extracted (920) from the volume of interest (VOI),and a VOI data structure is generated (930).

In many embodiments, the VOI is defined by a mask. In a variety ofembodiments, the VOI is defined by a masked that isanatomically-derived. Anatomically-derived masks can be spheres thatmask out specific regions of the brain. However, any shape of mask,including complex three-dimensional structures can be used asappropriate to the requirements of a given application. In numerousembodiments, the mask used to define the VOI masks out regions that arenot in a ROI. The eigenvariate of the time series from the VOI can beextracted using the SPM8 library. In some embodiments, extracting theeigenvariate further includes obtaining a specific task. In a variety ofembodiments, extracting the eigenvariate includes obtaining a specificmask. In many embodiments, extracting the eigenvariate includesobtaining a contrast number from a first level GLM. Generating a VOIdata structure can include a deconvolved time course containing only thephysiological component of the PPI, the psychological variabledescribing the parametric contrast of task onsets from the contrast ofinterest, and the interaction between the physiological andpsychological variables. This data structure can function as the PPImodel. In numerous embodiments, nuisance regressors are used from thefirst level GLM. While a specific method of generating a PPI datastructure has been described above, any number of additional variablescan be used as appropriate to the requirements of a given application.Connectivity preprocessed image data can be further utilized to generateresting state models. Methods for generating resting state models aredescribed below.

Generating Resting Models

Resting state models can be used to determine the default resting stateof the brain. The default resting state can give insight into the normallevel of reactivity for a particular region of the brain for theindividual in question. Hypo, typical, or hyper activation betweendifferent regions can give valuable insight into both brain relateddisorders, and potential treatment methods.

In many embodiments, resting state models are acquired by performingfMRI scans on a patient who is instructed to perform no tasks, and isnot provided any stimuli. In a variety of embodiments, resting statemodels can be generated using fMRI data acquired during a test battery.Residual images can be generated using the test battery fMRI data, whichcan be used to generate a model of the patient's resting state. Imagedata acquired during various tasks can be useful in helping isolateneural activation observed during intervals in which tasks are not beingperformed, but that is task related. In this way, neural circuitpathways activated during resting state can be better separated fromactivations that are artifacts arising from the challenges ofmaintaining patients in a true resting state during imaging. Whileremoval of such artifacts typically results in more informative neuralcircuit pathway classification, neural circuit pathway classificationcan be performed using processes without, or with minimal artifactremoval.

Turning now to FIG. 10, a process for generating resting state models inaccordance with an embodiment of the invention is illustrated. Process1000 includes obtaining (1010) connectivity preprocessed image data, andconcatenating (1020) preprocessed image data across tasks. Segmentationis performed (1030), and a task and nuisance regressors matrix isgenerated (1040). Residual images are generated (1050) using theregressors matrix, and the residual images are filtered (1060). Voxelwise resting state models are generated (1070) and ROI resting statemodels are generated (1080).

In many embodiments, time-series data is generated for each taskprovided to the patient. Time-series data can be concatenated to form asingle time-series image sequence. In many embodiments, concatenation isperformed by appending successive time-series to a chosen first timeseries. Segmentation can be performed on the time-series in order tosegment the structural images into white matter, grey matter, andcerebrospinal fluid (CSF) probability maps. In many embodiments,segmentation is performed using the FAST program from the FSL library.In order to optimize segmentation, white matter and CSF masks can beeroded from the edges inward for several iterations in order to ensurethat the masks are capturing correct tissue matter. In a variety ofembodiments, erosion can be performed using the SPM library. In manyembodiments, three iterations are applied, however any number ofiterations can be applied in accordance with the requirements of a givenapplication. The segmented time-series sequence of image data can bewarped into coordinate space defined by the Montreal NeurologicalInstitute (MNI space), and additional masking can be applied to finetune segmentation. For example, a mask can be used to remove additionalgrey matter from the white matter segment as a result of the erosion. Inmany embodiments, the mask is an atlas from the AAL Single Subject Atlasset provided by the Montreal Neurological Institute. Once segmentationhas been performed, the mean time-series for each segment can be storedand/or provided.

A regressor matrix can be generated for use with the segmentedtime-series. In many embodiments, the regressor matrix includes some orall of: task regressor information, including a constant to indicateoverall task effects; spike regressor information; realignmentparameters, and their temporal derivatives and their squares; and whitematter and CSF time-series across all tasks. Residual images can begenerated by running regression to move the variance associated with aparticular task, movement, and white matter/CSF noise using thegenerated regressor matrix. Residual images can then be filtered bybandpass filtering. In many embodiments, a high frequency cutoff of0.009 Hz and a low frequency cutoff of 0.08 Hz is used. However,different cutoffs can be used as appropriate to the requirements of agiven application.

Resting state models can be generated using the filtered images. Voxelwise resting state models can be generated by extracting the time-seriesfor a specific task, and regressing against the time-series from allvoxels in the brain. Fisher tests, Z-transforms, and/or a time-seriesacross a brain atlas can be generated, indicating the overall restingstate for each voxel. ROI resting state models can be generated by usingthe DPABI toolbox by The R-fMRI Network, which can be found athttp://rfmri.org/dpabi. As noted above, the ability to develop restingstate models using information concerning task related activation ofvoxels can assist with removal of artifacts resulting from thechallenges of a patient maintaining a true resting state during imaging.In many embodiments, however, resting state models are built using onlydata obtained while a patient is not receiving stimuli.

Resting state models can be used to identify neural circuit pathwaysthat correspond to biotype classifications. In many embodiments,specific patterns of reactivity and connectivity can be used to classifythe presence of a specific complex in the observed data. In the contextof MRI data the specific complex network classifications can be referredto as biotype classifications, and medication information to treat agiven reactivity imbalance can be issued using the image processingsystem. An example of biotype classification is discussed below.

Biotype Classification

Neural circuit pathway circuits revealed by image processing systems inaccordance with various embodiments of the invention can be used toautomatically identify and/or classify the presence of complex networksthat can be referred to as biotypes. In numerous embodiments, machinelearning is used to automatically identify and/or perform biotypeclassification based upon observed resting state neural pathwaycircuits. In some embodiments, a support vector machine is used toautomatically perform biotype classification. In several embodiments,convolutional neural networks are used to perform biotypeclassification. A database can be used to train machine learning systemsto identify and classify specific biotypes based on observed neuralactivation and connectivity. In many embodiments, the database is madeof MRI image data annotated with complex network classifications, and/orany other type of training data can be used as appropriate to therequirements of a given application. Data in the database can becollected using standardized data collection procedures, including, butnot limited to, standardized stimuli order, standardized stimuli time,standardized image processing, and/or any other standardizationtechnique as appropriate to the requirements of a given application.

Biotype classifications can be assigned by analyzing the connectivityand reactivity between regions of the brain. In many embodiments,resting state models are used to assign biotype classifications. Restingstate models can contain a set of numbers representing the strength ofconnection between regions of the brain. Regions of the brain can beassociated with various neural circuits. In numerous embodiments,approximately 50 regions are selected for analysis, however a larger orsmaller number of regions can be used in accordance with therequirements of a given application. The average strength of theconnections over the selected regions are compared to baselinemeasurements. Based on the average strength of the connections, adetermination can be made as to whether or not the neural circuitry isbehaving normally or abnormally. In a variety of embodiments,abnormality is determined using a threshold. The threshold can be aspecific strength, a predefined number of standard deviations from thenorm, and/or any other criterion as appropriate to the requirements of agiven application.

Normal and/or abnormal circuits can be used to assign at least onebiotype. In many embodiments, assigning biotype classifications usingabnormal circuits is performed using a heuristic. In some embodiments, aranking system is used. In a variety of embodiments, a machine learningalgorithm is used. The machine learning algorithm can learn how toweight the importance of specific regions when determining how to assignbiotypes. In a number of embodiments, a profile approach is utilized. Aprofile approach can include matching a patient's observed neuralcircuit pathways to a database of known neural circuit pathway profiles.A patient may be assigned one or more biotypes. In many embodiments,patients neural circuit pathways can appropriately be classified as anamalgam of biotype classifications. In some embodiments, the imageprocessing system provides the biotype classifications that a patient'sresting state neural circuit pathways most closely matches. Provision ofbiotype classifications can include an indicator of fit, indicating howclosely the patient matches a given biotype. The indicator of fit can bea percentage, a statistical estimate, or any other indicator inaccordance with the requirements of a given application.

Connections between various regions in the brain that are responsiblefor the brain performing a set of different physiological and/orpsychological functions can be defined as neural circuit pathways.Neural circuit pathways can be mapped and stored in a taxonomy, and/orbrain atlas. Turning now to FIG. 11, a set of example neural circuitpathways are illustrated in accordance with an embodiment of theinvention. Deviations in reactivity in specific regions, and/orconnectivity can disrupt the normal function of the neural circuitpathways. Hypo, typical, and hyper activity and/or connectivity statescan be correlated to specific neurological issues that can be expressedas mental disorders. Turning now to FIG. 12, a chart illustrating theinterplay between connectivity and reactivity in accordance with anembodiment of the invention is illustrated. Brain images showing theneural circuitry for the default mode, salience, negative affect,positive affect, attention, and cognitive control are illustrated, aswell as example conditions (rumination, anxious avoidance, negativebias, threat dysregulation, anhedonia, context insensitivity,inattention, and cognitive dyscontrol) and their respective abnormalconnectivity and reactivity are illustrated. While specific neuralcircuit pathways are illustrated in FIG. 12, numerous neural circuitpathways exist as complex networks within the brain and can beidentified and/or assigned in accordance with the requirements of agiven application.

Image processing systems can generate image data representing apatient's brain to produce similar outputs using reactivity andconnectivity models. Patient biotype can further be displayed as a radarplot as illustrated in FIG. 13. In many embodiments, scores can begenerated that represent the various connections and reactions in neuraltissue. In a variety of embodiments, scores can be generated whichrepresent the various circuits that cause particular symptoms. Innumerous embodiments, data, such as scores, obtained images, processedimages, biotype classifications, and/or any other type of data collectedor generated by image processing systems can be provided to the user. Ina variety of embodiments, data is provided via a user interface device.However, any number of devices and methods can be used to provide datato a user in accordance with a given application. Turning now to FIGS.14, 15A, 15B, and 15C, scoring sheets in accordance with an embodimentof the invention are illustrated. In some embodiments, scores can beaccompanied by a description of the particular biotype. In manyembodiments, score reports and biotype assignment are automaticallyperformed by the image detection system based on the connectivity andreactivity data. In numerous embodiments, only a resting state fMRI isneeded to assign biotypes. Based on assigned biotypes, specific drugsand/or drug classes can be recommended to regulate dysfunction caused bythe abnormal neural circuitry. Methods for assigning drugs are describedbelow.

Assigning Drugs Based on Biotype

In many embodiments, specific complex networks within the brain (i.e.biotype classifications) as identified by the image processing systemcan be used to assign drugs to remediate the patient. Different drugsand drug classes affect different chemical issues within the brain. Byidentifying the abnormal circuitry, and how it is behaving abnormally,specific drugs therapies and mental health therapeutic techniques can berecommended in order to target the specific neural circuit pathwayslikely responsible for particular behavioral tendencies. Turning now toFIG. 16, examples of specific therapies that affect specific neuralcircuits in accordance with an embodiment of the invention areillustrated. In many embodiments, the image processing system can beintegrated with a drug database that describes the function ofparticular psychoactive medication. Image processing systems can assigndrugs based on their physiological effects to remediate the identifiedabnormal circuitry. In some embodiments, machine learning is utilized torefine the effect of particular therapies on biotypes, and/or amalgamsof biotypes. In some embodiments, therapies can be stored in a database,such as, but not limited to, a brain atlas, a taxonomy, a drug database,or any other data structure as appropriate to the requirements of agiven application. In many embodiments, therapies stored in a databaseare associated with at least one biotype and/or neural circuit.

In numerous embodiments, doctors treat patients using methods suggestedby image processing systems. In a variety of embodiments, patients arescanned using MRI machines. The resulting scan data can be used by imageprocessing systems to assign biotypes. Biotypes can be associated withone or more treatments and/or drug therapies, which can be administeredto the patient. In numerous embodiments, biotypes and/or scores can beused to generate an efficacy metric describing efficacy of a particulardrug and/or class of drugs on the particular patient. Efficacy metricscan be used to predict treatment outcomes for patients for particulartreatments. In numerous embodiments, efficacy metrics are used torecommend a particular treatment with the highest likelihood of success.

Although the present invention has been described in certain specificaspects, many additional modifications and variations would be apparentto those skilled in the art. In particular, any of the various processesdescribed above can be performed in alternative sequences in order toachieve similar results in a manner that is more appropriate to therequirements of a specific application. It is therefore to be understoodthat the present invention can be practiced otherwise than specificallydescribed without departing from the scope and spirit of the presentinvention. Thus, embodiments of the present invention should beconsidered in all respects as illustrative and not restrictive.

What is claimed is:
 1. A method for identifying complex networks intime-series sequence of image data, comprising: obtaining time-seriessequence of image data from an image capture device; realigning thetime-series sequence of image data to a fixed orientation using an imageprocessing server system; unwarping the time-series sequence of imagedata using the image processing server system; despiking the time-seriessequence of image data using the image processing server system;detecting objects within the time-series sequence of image data usingthe image processing server system; measuring connectivities betweendetected objects within the time-series sequence of image data using theimage processing server system; matching the measured connectivitiesbetween detected objects to at least one complex network stored in adatabase of complex networks using the processor; and displaying the atleast one matched complex network using a display device.